DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 09/05/2023, 02/28/2025, and 01/27/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-5, and 8-9 is/are rejected under 35 U.S.C. 102(a) as being unpatentable over Tachi et al. (U.S. Patent Publication No. 7659502-B2, hereinafter “Tachi”)
Regarding claim 1, Tachi teaches: An electronic apparatus comprising: a contact portion comprising a plurality of markers; (Description [0003], "The transparent elastic body has two kinds of marker groups embedded therein. When force is applied to the transparent elastic body from beneath, the marker moves from what is shown in the left diagram to what is shown in the right diagram.") a camera; (Description [0005], "The left diagram shows one unit comprising one CCD camera and an elastic body section, while the right diagram shows the result of area enlargement combining units.")and at least one processor configured to: (Description [0030], "The processor of the computer 8 calculates the marker information (movement vectors as movement information, for example) regarding the behavior (displacement, strain or inclination) of markers. The processor reconstructs the distribution of forces applied to the surface 2 by an object 5 using the marker information (movement information, for example) and a transfer function that is stored in the memory device of the computer 8.")
identify, based on at least one surface of the contact portion contacting an object, at least one displacement of the plurality of markers based on a first image in which a first surface of the contact portion that is opposite to the at least one surface is captured by the camera, (Description [0032], "With the present invention, a plurality of optical markers are distributed within the transparent elastic body 1, and information about a contacting object and information about displacement and deformation within the elastic body produced by contact are detected by photographing situations where displacement, deformation and inclination arise in the markers due to deformation of the elastic body 1 as a result of the object coming into contact with the elastic body 1 using a camera.")
and identify, based on the at least one displacement, at least one of an intensity of a force applied to the contact portion and a direction of the force applied to the contact portion, (Description [0042], " Here, eight components, f=[fx(1), fx(2), fx(3), fx(4), fz(1), fz(2), fz(3), fz(4)] are force vector distribution to be obtained, where m=[m(1), m(2), m(3), m(4)] and n=[n(1), n(2), n(3), n(4)] are movement vectors to be measured. The vectors m and n are represented as X. Namely, X=[m(1), m(2), m(3), m(4), n(1), n(2), n(3), n(4)]. Here, movement vectors m and n that are observed when a unit force (magnitude of 1) in the x-direction is applied to a point 1 are represented as Mx(1).")
wherein a first marker of the plurality of markers comprises at least one of a form and a color that is different from a second marker of the plurality of markers that is adjacent to the first marker. (Description [0038], "Here, in an environmental type tactile sensor which similarly employs a plurality of cameras, the stripe or black-white tile pattern with known pattern intervals is already arranged in a known position as a colored marker for sensing and ready to be acquired as an image.")
Regarding claim 2, Tachi teaches: The electronic apparatus of claim 1, wherein each of the plurality of markers comprises at least one first pattern of a first form and at least one second pattern of a second form that is provided at one side of the at least one first pattern, (Description [0083], "colored markers being comprised of extremely thin cylinders or columns having microscopic cross sections are shown. Two marker groups are arranged at different depths from the surface 2. A marker group made up of extremely thin blue cylindrical markers 40 and another marker group made up of extremely thin red cylindrical markers 30 are embedded along the surface 2 and are layered at different depths from the surface.")
and wherein the at least one processor is further configured to identify at least one of the intensity of the force and the direction of the force based on the at least one first pattern the at least one second pattern. (Description [0042], "Here, eight components, f=[fx(1), fx(2), fx(3), fx(4), fz(1), fz(2), fz(3), fz(4)] are force vector distribution to be obtained, where m=[m(1), m(2), m(3), m(4)] and n=[n(1), n(2), n(3), n(4)] are movement vectors to be measured. The vectors m and n are represented as X. Namely, X=[m(1), m(2), m(3), m(4), n(1), n(2), n(3), n(4)]. Here, movement vectors m and n that are observed when a unit force (magnitude of 1) in the x-direction is applied to a point 1 are represented as Mx(1).")
Regarding claim 3, Tachi teaches: The electronic apparatus of claim 2, wherein the first form comprises a circular pattern, and wherein the second form comprises a linear pattern. (Fig. 5; Description [0036], "Thereafter, the respective images from the cameras are integrated such that the markers in the overlapped photograph regions 11 are matched, thus integrating image data. In FIG. 5, a black circle represents a blue marker, while a white circle represents a red marker, and the photograph regions of the camera 1 and the camera 2 have an overlapped photograph region 11."; Examiner's Note - The second pattern of markers is set in a linear line.)
Regarding claim 4, Tachi teaches: The electronic apparatus of claim 3, wherein the at least one processor is further configured to identify a rotational force that is parallel to the contact portion based on displacement of the linear pattern. (Description [0042], "Here, eight components, f=[fx(1), fx(2), fx(3), fx(4), fz(1), fz(2), fz(3), fz(4)] are force vector distribution to be obtained, where m=[m(1), m(2), m(3), m(4)] and n=[n(1), n(2), n(3), n(4)] are movement vectors to be measured. The vectors m and n are represented as X. Namely, X=[m(1), m(2), m(3), m(4), n(1), n(2), n(3), n(4)]. Here, movement vectors m and n that are observed when a unit force (magnitude of 1) in the x-direction is applied to a point 1 are represented as Mx(1)."; Examiner's Note - Prior art can identify forces from rotational forces )
Regarding claim 5, Tachi teaches: The electronic apparatus of claim 2, wherein the at least one second pattern comprises a plurality of second patterns, and wherein adjacent patterns of the plurality of second patterns vary in color. (Description [0083], "colored markers being comprised of extremely thin cylinders or columns having microscopic cross sections are shown. Two marker groups are arranged at different depths from the surface 2. A marker group made up of extremely thin blue cylindrical markers 40 and another marker group made up of extremely thin red cylindrical markers 30 are embedded along the surface 2 and are layered at different depths from the surface."; Examiner's Note - Prior art has a plurality of red cylinders which maps to plurality of second patterns, and there are blue and red cylinders adjacent to any given red cylinder mapping to the plurality of patterns varying in color adjacent to the second pattern.)
Regarding claim 8, Tachi teaches: The electronic apparatus of claim 1, wherein each of the plurality of markers comprises:, a first pattern of a first form, or a first pattern of the first form and at least one second pattern of a second form adjacent to the first pattern of the first form. (Description [0038], "Here, in an environmental type tactile sensor which similarly employs a plurality of cameras, the stripe or black-white tile pattern with known pattern intervals is already arranged in a known position as a colored marker for sensing and ready to be acquired as an image.")
Regarding claim 9, Tachi teaches: The electronic apparatus of claim 1, further comprising: a memory configured to store a second image in which a surface of the contact portion is captured through the camera prior to the contact portion contacting the object, wherein the at least one processor is further configured to identify the at least one displacement based on the first image and the second image. (Description, [0030], "The processor of the computer 8 calculates the marker information (movement vectors as movement information, for example) regarding the behavior (displacement, strain or inclination) of markers. The processor reconstructs the distribution of forces applied to the surface 2 by an object 5 using the marker information (movement information, for example) and a transfer function that is stored in the memory device of the computer 8.")
Regarding claim 11, claim 11 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tachi further teaching on: A control method of an electronic apparatus, the method comprising (Description [0040], “The transformation from the marker information M to the force information F is obtained by an equation F=HM. Referring to FIG. 7 and FIG. 8, a method of reconstructing the force vector distribution from the marker information will now be described based on a method of obtaining the force vector distribution from the movement vectors of markers.”)
Regarding claim 12, claim 12 has been analyzed with regard to claim 2 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tachi further teaching on: A control method of an electronic apparatus, the method comprising (Description [0040], “The transformation from the marker information M to the force information F is obtained by an equation F=HM. Referring to FIG. 7 and FIG. 8, a method of reconstructing the force vector distribution from the marker information will now be described based on a method of obtaining the force vector distribution from the movement vectors of markers.”)
Regarding claim 13, claim 13 has been analyzed with regard to claim 3 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tachi further teaching on: A control method of an electronic apparatus, the method comprising (Description [0040], “The transformation from the marker information M to the force information F is obtained by an equation F=HM. Referring to FIG. 7 and FIG. 8, a method of reconstructing the force vector distribution from the marker information will now be described based on a method of obtaining the force vector distribution from the movement vectors of markers.”)
Regarding claim 14, claim 14 has been analyzed with regard to claim 4 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tachi further teaching on: A control method of an electronic apparatus, the method comprising (Description [0040], “The transformation from the marker information M to the force information F is obtained by an equation F=HM. Referring to FIG. 7 and FIG. 8, a method of reconstructing the force vector distribution from the marker information will now be described based on a method of obtaining the force vector distribution from the movement vectors of markers.”)
Regarding claim 15, claim 15 has been analyzed with regard to claim 5 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tachi further teaching on: A control method of an electronic apparatus, the method comprising (Description [0040], “The transformation from the marker information M to the force information F is obtained by an equation F=HM. Referring to FIG. 7 and FIG. 8, a method of reconstructing the force vector distribution from the marker information will now be described based on a method of obtaining the force vector distribution from the movement vectors of markers.”)
Regarding claim 16, claim 16 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tachi further teaching on: A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to (Fig. 3)
Regarding claim 17, claim 17 has been analyzed with regard to claim 2 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tachi further teaching on: A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to (Fig. 3)
Regarding claim 18, claim 18 has been analyzed with regard to claim 3 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tachi further teaching on: A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to (Fig. 3)
Regarding claim 19, claim 19 has been analyzed with regard to claim 4 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tachi further teaching on: A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to (Fig. 3)
Regarding claim 20, claim 20 has been analyzed with regard to claim 5 and is rejected for the same reasons of obviousness as used above as well as in accordance with Tachi further teaching on: A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to (Fig. 3)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tachi et al. (U.S. Patent Publication No. 7659502-B2, hereinafter “Tachi”)
Regarding claim 6, Tachi teaches: The electronic apparatus of claim 1, wherein the first marker comprises a circular first pattern and a linear first pattern oriented from the circular first pattern toward a first direction, wherein the second marker is adjacent to the first marker in a direction that is opposite the first direction, and the second marker comprises a circular second pattern and a linear second pattern oriented from the circular second pattern toward a second direction that is perpendicular to the first direction, wherein the plurality of markers further comprises a third marker and a fourth marker, wherein the third marker is adjacent to the second marker in a direction that is opposite the second direction, and the third marker comprises a circular third pattern and a linear third pattern oriented from the circular third pattern toward a third direction that is perpendicular to the second direction, and wherein the fourth marker is adjacent to the third marker in direction that is opposite to the third direction, and the fourth marker comprises a circular fourth pattern and a linear fourth pattern oriented from the circular fourth pattern toward a fourth direction that is perpendicular to the third direction. (Fig. 18; Fig. 19; Description [0086], “Referring to FIG. 18, two marker groups (a marker group comprising a plurality of thin red strips arranged in a row and a marker group comprising a plurality of thin blue strips arranged in a row) are aligned so that respective markers are orthogonal to each other, but the spatial arrangement relationship between the plurality of marker groups is not limited.”; Description [0087], "FIG. 19 shows a sensing part having a plurality of plane markers. The plane markers are normally concealed by concealment markers and each plane marker is partitioned into a plurality of portions having different colors for each portion, and the partitioned portions having the same color constitute each marker group. The plane markers and said concealment markers are provided and spaced with each other in the elastic body, and an arrangement is made such that said the markers are concealed by the concealment markers and not observed in a state where external force is not acting on the transparent elastic body. When shear strain arises, the positions of the concealment markers 6 and the colored markers 20 become offset, giving color.")
Prior art teaches circular markers in Fig. 19 and perpendicularly crossing markers in Fig. 18. Combining them would result in marker pattern that maps with the claimed pattern. Such modification would have allowed for a fourth axis beyond the 3 that currently exist in Fig. 19 for more accurate shear stress measurement.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tachi et al. (U.S. Patent Publication No. 7659502-B2, hereinafter “Tachi”) further in view of Lepora et al. (WO 2020/128430-A1, hereinafter “Lepora”) and Norieda et al. (U.S. Patent Publication No. 10234955-B2, hereinafter “Norieda”)
Regarding claim 7, Tachi does not teach: The electronic apparatus of claim 1, further comprising: a lens array provided at an opposite surface of the contact portion, the lens array comprising a plurality of lenses corresponding to the plurality of markers, wherein the at least one processor is further configured to:
identify a size change of each of the plurality of markers through each of the plurality of lenses, and identify a force in a direction perpendicular to the contact portion based on the identified size change.
However, Lepora teaches: The electronic apparatus of claim 1, further comprising: a lens array provided at an opposite surface of the contact portion, the lens array comprising a plurality of lenses corresponding to the plurality of markers, wherein the at least one processor is further configured to: (Lepora, 13-14 pg. "The sensor 1 is provided with a lens 14 to aid the focusing of light from the tactile elements 12 on the imaging device 20. The sensor 1 is also provided with illumination means 15 operable to illuminate the tactile elements 12. In the present example, the illumination means 15 comprises one or more LEDs mounted on a ring provided around the imaging device 20 aperture. The skilled man will appreciate that in other embodiments different combinations of lenses 14 and/or illumination means 15 may be provided.").
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify Tachi to include imaging device with multiple lenses and focuses as taught by Lepora because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, optical tactile sensor (Tachi) as modified by imaging device with multiple lenses and focuses (Lepora) can yield a predictable result of allowing for multiple aperture lengths and lens features since having multiple lenses allow for variable lens usage and variable focal lengths. Thus, a person of ordinary skill would have appreciated including in optical tactile sensor (Tachi) the ability to have multiple lenses and focuses since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Tachi in view of Lepora does not teach: identify a size change of each of the plurality of markers through each of the plurality of lenses, and identify a force in a direction perpendicular to the contact portion based on the identified size change.
However, Norieda teaches: identify a size change of each of the plurality of markers through each of the plurality of lenses, and identify a force in a direction perpendicular to the contact portion based on the identified size change. (Norieda, [0085] "For example, the position calculation unit 2020 holds information regarding a shape, a size, a color, and the like of the marker in advance, detects the marker from the acquired captured image 22 by using the known information, and calculates a position thereof. ").
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify Tachi in view of Norieda to include marker shape and size identification because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, optical tactile sensor (Tachi in view of Lepora) as modified by shape and size identification (Norieda) yield a predictable result of allowing easier distance and angle calculation since as the distance between sensor and marker gets greater the relative size of the marker in the image gets smaller and as the angle of the sensor to the marker grows the deformation of the shape also grows. Thus, a person of ordinary skill would have appreciated including in optical tactile sensor (Tachi in view of Lepora) the ability to do shape and size identification (Norieda) since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tachi et al. (U.S. Patent Publication No. 7659502-B2, hereinafter “Tachi”) further in view of Lepora et al. (WO 2020/128430-A1, hereinafter “Lepora”)
Regarding claim 10, Tachi does not teach: The electronic apparatus of claim 1, wherein the contact portion comprises: a transparent gel; and a coating layer formed at an upper end of the plurality of markers, and wherein the plurality of markers are provided at one surface of the transparent gel.
However, Lepora does teach: The electronic apparatus of claim 1, wherein the contact portion comprises: a transparent gel; and a coating layer formed at an upper end of the plurality of markers, and wherein the plurality of markers are provided at one surface of the transparent gel. (Lepora, pg. 14, "The chamber may be filled with a suitable gas, liquid or gel. In a preferred embodiment, the space is filled with an optically clear gel. The gel filling may aid reversion to the neutral configuration and/or damp minor tactile events.")
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify Tachi to include the chamber being filled with gel because such a modification is the result of combining prior art elements according to known methods to yield predictable results. More specifically, optical tactile sensor (Tachi) as modified by the chamber being filled with gel (Lepora) can yield a predictable result of allowing for neutral configuration to be easily returned to when no input is being applied. (Lepora, pg. 14, “The gel filling may aid reversion to the neutral configuration and/or damp minor tactile events.”) Thus, a person of ordinary skill would have appreciated including in optical tactile sensor (Tachi) the ability to have the chamber filled with gel since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jinsu Hwang whose telephone number is (703)756-1370. The examiner can normally be reached Mon 6am-8am, 3pm-9pm EST; Thu 12pm - 8pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JINSU HWANG/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667